Information drives business, education, government and the like. Individuals, companies, institutions and other entities rely to an unprecedented extent upon online, frequently accessed, constantly changing data. Events that inhibit the availability and accuracy of such data may have negative consequences for such entities. Replication, backup and continuous data protection techniques are utilized to minimize data loss and improve the availability of data. In the event of a failure of one or more physical disks for storing data, or failure of a node or host data processing system associated with such a disk, the data on a remote node may not be utilized.
Data storage is typically allocated from one or more storage devices that are maintained as a “volume.” The “volume” may serve as a logical interface used by an operating system to access data stored on one or more storage media using a single instance of a file system. Thus, a volume may act as an abstraction that essentially “hides” storage allocation and (optionally) data protection and/or redundancy from the application. An application can store its data on multiple volumes. The content of a volume is accessed using fixed sized data units called blocks. Replication may be implemented at the file level or the block level.
Periodic replication is an important feature of file store solutions. A snapshot or checkpoint operation is often used to avoid downtime. In Veritas File System (VxFS), a checkpoint mechanism is used to track changes and to provide stable copy for reading the same. However, performing a replication operation, backup operation, or the like on a large data set may take a significant amount of time to complete. When replicating using checkpoints, the file system is locked, so that other accesses to the system are blocked, to make a copy for use in generating the checkpoint. Each conventional checkpoint is a copy of the entire file system. The replication is then implemented by comparing two checkpoints to determine the difference between the two. The difference is then copied to the remote node. In addition, the read-copy-modify cycle used for writes to a file system impact the performance on the server which is being replicated. In particular, for every write on the file system, a read of the file system and a write to the backup has to be performed for all files in the file system being modified at the time. The write is performed twice if the file system is also using the checkpoints to track the changes. If checkpoints are not used to track changes, then either the whole file system is scanned to track the changes or fetch command load overhead is incurred. If the increments at block level (BLIB) are taken, then the checkpointing overhead is incurred all the time. Accordingly, there is a continuing need for improved data replication techniques.